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Overview 


The  Office  of  the  Controller,  City  Services  Auditor,  has  completed  its  review  of  a  selection  of 
nonprofit  agencies  that  hold  contracts  with  the  City.  This  review  is  one  element  of  a  three-part 
effort  to  measure  and  improve  performance  both  of  the  City's  nonprofit  contract  management 
function  and  of  the  health  and  human  services  themselves  that  are  provided  by  nonprofit  agencies. 
The  other  two  parts  are:  technical  assistance  and  training  for  nonprofits,  and  implementation  of 
recommendations  by  the  City's  Nonprofit  Task  Force  that  the  City  streamline  its  monitoring  of 
nonprofit  contractors. 

Further  information  and  resources  related  to  each  of  these  efforts,  including  the  text  of  the 
Controller's  Finance  Guide  for  Nonprofit  Organizations,  can  be  found  on  our  website  at: 
http://www.sfgov.org/site/controller  index.asp?id=30547. 

In  addition  to  these  performance  measurement  and  management  efforts,  the  City  Services  Auditor 
recently  completed  fiscal  and  compliance  reviews  of  22  nonprofit  agencies  to  ensure  San 
Francisco's  compliance  with  requirements  under  the  federal  Single  Audit  Act  of  1984,  _as 
amended.  This  law,  as  implemented  by  OMB  Circular  No.  A-133,  requires  audits  of  states,  local 
governments  and  nonprofit  organizations  expending  federal  awards.  Further  information  and 
results  from  those  monitoring  site  visits  will  be  available  in  a  forthcoming  report  from  the 
Controller's  Office. 

Scope  and  Methodology 

We  began  this  review  with  an  effort  to  conduct  combined  financial  and  performance  audits  of  a 
sample  group  of  nonprofit  contractors.  The  overall  objective  was  to  improve  the  quality  of  the 
City's  contracted  health  and  social  services  by  adding  an  audit  process  to  the  tools  that  San 
Francisco  uses  to  ensure  that  resources  in  this  area  being  spent  appropriately.  We  selected  a  sample 
group  of  nonprofit  providers,  and  at  each  provider  the  objectives  were  to  determine  if  the  agency 
has  appropriate  financial  controls  and  meets  the  performance  requirements  in  its  City  contracts. 

To  arrive  at  a  sample  group,  we  listed  nonprofit  agencies  that  had  contracts  or  grants  with  at  least 
one  of  four  City  departments:  the  Department  of  Public  Health,  the  Human  Services  Agency,  the 
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Department  of  Children,  Youth  and  Their  Families,  and  the  Mayor's  Office.  Most  of  the  agencies 
worked  with  two  or  more  of  these  departments.  We  chose  to  focus  on  contracts  of  the  largest  two 
of  these  departments,  the  Department  of  Public  Health  and  the  Human  Services  Agency.  We 
interviewed  contract  managers  at  these  two  departments  about  issues  of  concern  to  the  City  and,  as 
a  result,  listed  the  types  of  services  provided  by  57  nonprofit  agencies.  We  selected  six  agencies 
and  conducted  financial  and  performance  audit  surveys  with  each.  We  considered  one  additional 
nonprofit  contractor,  however,  we  determined  that  that  provider  would  be  more  appropriately 
served  through  technical  assistance  and  training. 

The  Controller's  Office's  survey  process  included  the  following: 

•  Reviewed  the  most  recent  financial  audit,  and  management  letter  if  any,  of  each 
organization  by  its  certified  public  accountant; 

•  For  four  agencies,  reviewed  the  workpapers  of  external  auditors  to  determine  that  they  had 
developed  an  understanding  of  the  control  environment  and  had  tested  the  organization's 
internal  controls  around  accounting,  billing,  and  payroll  practices; 

•  Worked  with  the  chief  financial  officer  and/or  accounting  staff  at  three  agencies  to  review 
accounting,  billing  and  payroll  practices  at  a  high  level  and  to  review  the  agency's  chart  of 
accounts  and  cost  allocation  methods; 

•  Reviewed  the  City's  contracts  with  each  agency  and  reports  related  to  contract  performance 
such  as  monitoring  reports  and  units  of  service  reports,  and; 

•  Visited  each  agency  and,  in  some  cases,  interviewed  program  managers  and  reviewed 
client  materials. 

After  the  survey  process  and  follow-up  work  we  determined  that  that  we  could  make  limited 
findings  regarding  the  financial  controls  and  contract  performance  at  the  agencies  and  also  that  full 
financial  and  performance  audits  would  not  benefit  the  City  sufficiently  to  justify  the  time  and  cost 
required.  We  present  our  findings  here  and  have  also  reported  them  directly  to  the  agencies.  In 
addition,  we  identified  and  are  reporting  on  common  areas  of  concern  regarding  the  City's 
contracts  that  will  be  of  use^  to  the  Department  of  Public  Health  and  the  Human  Services  Agency 
and  to  San  Francisco's  nonprofit  contracting  efforts  generally. 

Asian  American  Recovery  Services  (AARS) 

AARS  is  a  provider  of  substance  abuse  programs  for  the  Asian  and  Pacific  Islander  communities 
of  the  San  Francisco  Bay  Area  and  is  a  fiscal  intermediary  for  the  City's  Department  of  Public 
Health  (DPH).  As  of  June  2005,  the  value  of  all  contracts  (including  multi-year  contracts)  between 
AARS  and  DPH  was  $9.8  million,  of  which  $3.3  million  was  for  fiscal  year  2004-05. 

AARS  appears  to  be  adequately  administering  its  contracts  with  DPH  for  fiscal  intermediary 
services  at  City  health  clinics.  The  total  value  of  these  fiscal  intermediary  contracts  was  $3.8 
million  as  of  fiscal  year  2003-04,  the  most  recent  year  for  which  independently  audited  financial 
records  were  available  when  we  conducted  our  work.  Normally,  a  fiscal  intermediary  is  an  agency 
that  acts  as  the  go-between  for  the  City  and  a  smaller  agency  that  provides  a  service  but  does  not  or 
cannot  handle  the  financial  side  of  the  contract's  administration.  In  this  case,  however,  AARS 
provides  its  employees  to  work  in  DPH  clinics. 
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We  interviewed  AARS  staff  and  examined  financial  and  personnel  records  and  supporting 
documentation  provided  by  AARS  related  to  the  outcome  objectives  in  its  fiscal  intermediary 
services  contracts  with  DPH.  Concerning  these  objectives,  we  found  the  following: 

•  Invoices  that  AARS  submitted  to  DPH  for  payment  reflect  actual  costs  per  the  financial 
records  and  audited  financial  statements  of  AARS. 

•  Source  documents  support  costs  that  AARS  reported  on  invoices. 

•  AARS  maintains  adequate  internal  accounting  controls  to  properly  record  and  report 
transactions  and  to  safeguard  assets. 

•  Personnel  files  contain  documentation  showing  that  AARS  observes  all  legal  employment 
obligations  and  complies  with  the  management  policies  outlined  in  its  employee  manual. 

We  reviewed  the  audited  financial  statements  of  AARS  for  the  period  of  July  1,  2003  to  June  30, 
2004,  and  found  that  the  independent  auditor  rendered  an  unqualified  opinion  and  a  report  on 
compliance  and  internal  control  over  financial  reporting  that  noted  no  material  weaknesses.  As  part 
of  our  evaluation  of  internal  controls,  we  reviewed  the  audit  workpapers  of  the  independent 
auditor.  We  determined  that  the  auditor  had  engaged  in  adequate  planning  and  had  gained  an 
understanding  of  internal  controls  at  AARS  and  had  performed  sufficient  tests  of  transactions  to 
conclude  that  internal  controls  were  in  place. 

We  reviewed  six  recent  annual  monitoring  reports  prepared  by  DPH,  each  concerning  a  different 
AARS  program  in  San  Francisco: 

•  Asian  Drug  Abuse  Prevention  and  Treatment  (Project  ADAPT) — Intensive  outpatient 
treatment  services  for  adults. 

•  Comprehensive  Outreach  Program  for  Pacific  Islander  and  Asian  Substance  Abusers 
(COPPASA) — Street  outreach  primarily  to  Asian  and  Pacific  Islanders  who  are  at-risk  for 
substance  abuse  or  HIV  infection. 

•  COPPASA  Ancillary  Case  Management — Outreach  and  case  management  for  incarcerated 
Asian  and  Pacific  Islanders  to  help  them  find  treatment  programs  and  HIV  services  to  at- 
risk  individuals  not  in  jail. 

•  Lee  Woodward  Counseling  Center — Intensive  outpatient  substance  abuse  treatment 
services  for  women,  their  children,  their  families  and  /or  their  significant  others. 

•  Personal  Assisted  Employment  Services  (PAES)  Counseling  Services — Intensive 
behavioral  health  outpatient  program  for  adults  that  provides  integrated  mental  health  and 
substance  abuse  services  to  PAES  participants. 

•  Residential  Therapeutic  Community — Substance  abuse  counseling  and  treatment  for  up  to 
19  adults  who  are  housed  for  at  least  six  months  in  a  facility  at  Hayes  and  Cole  streets. 

Four  of  the  reports  gave  program  performance  ratings  of  commendable/exceeds  standards,  one 
gave  a  rating  of  acceptable/meets  standards,  and  one  gave  a  rating  of  needs  improvement/below 
standards.  The  program  rated  below  standards  for  fiscal  year  2003-04,  the  Lee  Woodward 
Counseling  Center,  was  assigned  to  AARS  by  the  City  when  the  organization  that  had  run  it, 
Women  and  Children's  Family  Services,  was  dissolving.  According  to  DPH,  since  that  report  was 
produced  AARS  hired  a  new  program  director  that  significantly  improved  the  quality  of  the 
program.  We  interviewed  DPH  staff  including  program  officers  Po  Yee  Lindahl  and  Sue  Marshall, 
who  had  positive  comments  about  AARS'  programs. 
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Community  Housing  Partnership  (CHP) 

CHP  provides  permanent,  affordable  supportive  housing  and  job  training  opportunities  to  formerly 
homeless  people  in  San  Francisco.  As  of  June  2005,  CHP  held  $3.5  million  in  City  contracts, 
consisting  of  three  three-year  contracts  spanning  July  2002  through  June  2007. 

CHP  appears  to  have  adequate  internal  financial  controls  to  ensure  that  it  is  properly  administering 
its  contracts  with  the  City's  Human  Services  Agency  (HSA)  and,  according  to  HSA  monitoring,  is 
performing  satisfactorily  under  these  contracts.  We  reviewed  the  audited  financial  statements  of 
CHP  for  the  period  of  July  1,  2003  to  June  30,  2004,  and  found  that  CHP's  independent  auditor 
rendered  an  unqualified  opinion  and  a  report  on  compliance  and  internal  control  over  financial 
reporting  that  noted  no  material  weaknesses.  We  examined  workpapers  prepared  by  CHP's 
independent  auditor  to  support  its  conclusions  concerning  CHP's  internal  controls.  We  determined 
that  the  auditor  had  engaged  in  adequate  planning,  had  gained  an  understanding  of  the  control 
environment,  and  had  performed  sufficient  tests  of  transactions  to  conclude  that  internal  controls 
were  in  place.  We  also  interviewed  CHP  staff  and  reviewed  source  documents  to  learn  how  CHP 
records  and  reports  results  for  the  service  and  outcome  objectives  in  its  contracts  with  HSA. 

We  considered  the  major  programs  CHP  operated  under  contract  to  HSA  in  fiscal  year  2004-05: 

•  Tenant  Services — Optional  supportive  services  including  intake  and  assessment, 
counseling,  case  management,  and  life  skills  activities  to  formerly  homeless  adults  and 
families  living  at  four  supportive  housing  sites  operated  by  CHP:  Iroquois  Hotel,  Senator 
Hotel,  San  Cristina  Hotel  and  Island  Bay  Homes. 

•  Supportive  Housing  Employment  Collaborative — A  CHP-led  collaboration  of  five 
nonprofit  agencies  that  provides  formerly  homeless,  disabled  people  with  vocational 
training,  workplace  education,  employment,  and  job-retention  services. 

•  Integrated  Services  Network — Employment  training  and  on-the-job  experience  for 
formerly  homeless  individuals  who  reside  in  any  of  20  permanent  supportive  housing  sites 
operated  by  any  of  the  seven  nonprofit  agencies  that  are  part  of  the  network. 

We  reviewed  the  most  recent  HSA  monitoring  report  for  each  of  the  programs  above,  and  found 
that  all  three  gave  overall  ratings  of  acceptable/meets  standards.  Further,  we  interviewed  HSA  staff 
including  program  monitors  Robert  Hays  and  Shelagh  Little,  who  had  positive  comments  about 
CHP's  programs.  Based  on  our  limited  review  of  selected  program  performance  reports  for  the 
three  programs  and  interviews  of  CHP's  program  directors,  we  found  CHP's  methods  for 
collecting  the  performance  data  it  reports  to  HSA  were  generally  satisfactory. 

Richmond  Area  Multi-Services  (RAMS) 

RAMS  provides  a  variety  of  mental  health  services  to  children  and  adults,  many  of  which  are 
funded  via  contacts  with  DPH.  As  of  June  2005,  RAMS  held  contracts  with  DPH  worth  $16.3 
million,  $6.6  million  of  which  was  for  fiscal  year  2004-05. 

We  reviewed  the  audited  financial  statements  of  RAMS  for  the  period  of  July  1,  2003  to  June  30, 
2004,  and  found  that  the  independent  auditor  rendered  an  unqualified  opinion  and  a  report  on 
compliance  and  internal  control  over  financial  reporting  that  noted  no  material  weaknesses.  We 
also  audited  some  of  RAMS's  financial  records  and  concluded,  for  the  periods  we  tested,  that: 
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•  Payroll  costs  were  mostly  supported  by  payroll  reports  but  RAMS  employees  do  not 
always  have  functional  time  sheets,  and  the  allocation  of  employee  salaries  to  each  RAMS 
program  may  not  be  based  on  the  actual  hours  the  employees  worked  on  each  contract. 
However,  RAMS  does  pre-authorize  the  number  of  hours  each  employee  is  to  work  on 
each  contract  and,  according  to  its  chief  financial  officer,  RAMS  adjusts  the  wages  and 
benefits  charged  to  contracts  accordingly  if  actual  time  worked  does  not  agree  with  the  pre- 
authorized  hours. 

•  RAMS  billed  operating  expenditures  to  DPH  that  were  less  than  or  equal  to  those 
supported  by  RAMS's  records  and  allowable  under  its  contracts  with  DPH. 

•  RAMS's  indirect  costs  (for  example,  rent,  utilities,  repairs,  and  payroll  taxes  and  fees) 
billed  to  DPH  appear  reasonable  based  on  our  payroll  tests.  RAMS  did  not  correctly 
allocate  some  of  its  indirect  costs  among  its  programs;  however,  the  effect  was  immaterial 
for  fiscal  year  2003-04.  According  to  RAMS's  chief  executive  officer,  the  agency 
implemented  a  procedure  in  fiscal  year  2004-05  that  correctly  allocates  indirect  costs. 

•  The  audited  financial  statements  support  the  expenditures  that  RAMS  reported  on  its  cost 
reports  to  DPH,  and  indicate  that  the  fiscal  health  of  RAMS  has  improved  since  fiscal  year 
2002-03. 

We  also  reviewed  two  recent  monitoring  reports  prepared  by  DPH,  each  concerning  a  different 
major  RAMS  program  or  group  of  programs: 

.  •    Outpatient  and  Early  Periodic  Screening,  Detection  and  Treatment — A  comprehensive 
report  addressing  all  of  RAMS's  mental  health  services  for  adults  and  children. 

•  Bridge  to  Wellness  Psychiatric  Outpatient  Services — Partial  hospitalization  and  intensive 
outpatient  services  to  reduce  the  length  of  inpatient  stays  and  bridge  the  gap  between 
inpatient  and  outpatient  treatment. 

We  found  that  both  of  the  reports  gave  average  ratings  of  commendable/exceeds  standards. 
Further,  we  interviewed  DPH  staff  including  program  monitors  Edwin  Batongbacal  and  Albert 
Eng,  and  Khalil  Habeeb,  finance  director  for  the  SFGH  Department  of  Psychiatry.  All  had  positive 
comments  about  RAMS's  programs. 

Finally,  we  determined  whether  RAMS  had  achieved  the  targets  for  a  small,  purposefully  selected 
sample  of  performance  measures  set  in  its  adult  and  children's  services  contracts  with  DPH  for 
fiscal  year  2004-05.  Out  of  92  items  labeled  as  performance  objectives  in  these  two  contracts, 
about  half  (47)  had  measurable  targets.  Of  these  47  performance  measures,  we  purposefully 
selected  and  tested  7  and  found  RAMS  did  not  meet  the  target  for  6  of  them.  However,  for  2  of 
these  6  measures,  RAMS  fell  just  short  (target  was  50%  in  both  cases  and  RAMS  achieved  44% 
and  47%).  Two  other  measures  of  the  6  concern  the  Employment  Services  Program  that  RAMS 
operates  under  a  contract  with  the  state  Department  of  Rehabilitation.  (Because  DPH  and  DOR  are 
parties  to  a  cooperative  agreement,  this  program  is  included  in  RAMS's  contract  with  DPH.) 
According  to  RAMS's  chief  executive  officer,  management  personnel  of  RAMS,  DPH  and  the 
state  met  in  late  2005  and  the  parties  agreed  that  the  targets  for  this  program  are  unrealistic  and 
will  be  reduced.  For  a  fifth  measure  of  the  6,  RAMS  told  us  that  its  staff  appears  to  have  recorded 
some  incorrect  dates,  noting  the  dates  clients  chose  to  have  appointments  instead  of  earlier  dates 
on  which  appointments  were  offered  by  RAMS  and  could  have  occurred.  This  practice  would  give 
unfavorable  results  for  RAMS's  timeliness  in  offering  appointments  in  its  children's  outpatient 
mental  health  services  program. 
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Shanti 

Shanti  provides  education,  practical  assistance,  and  emotional  support  to  people  in  need,  focusing 
on  those  living  with  HIV/ AIDS  and  breast  cancer.  As  of  June  2005,  Shanti  held  $2.0  million  in 
contracts  with  DPH,  of  which  $1.3  million  was  for  fiscal  year  2004-05. 

We  reviewed  the  audited  financial  statements  of  Shanti  for  the  periods  of  July  1,  2003  to  June  30, 
2004  and  July  1,  2004  to  June  30,  2005,  and  found  that  Shanti's  independent  auditor  rendered 
unqualified  opinions  and  reports  on  compliance  and  internal  control  over  financial  reporting  that 
noted  no  material  weaknesses.  We  documented  key  indicators  of  Shanti's  fiscal  health  based  on  its 
audited  financial  statements  and  found  that  Shanti's  fiscal  health  and  fiscal  operations  have 
improved  since  fiscal  year  2003-04. 

We  also  reviewed  seven  recent  annual  monitoring  reports  prepared  by  DPH,  each  concerning  a 
different  Shanti  program  in  San  Francisco: 

Lifelines — Enhances  the  quality  of  life  of  underserved  women  with  breast  cancer  by 
providing  intake,  support  from  peer  volunteers  and  transportation  services. 
Van  Transportation  Services — Took  people  with  HIV/AIDS  to  medical  and  social  services 
appointments,  and  provided  light  moving  services.  Due  to  funding  cuts,  the  service  was 
discontinued  after  June  2004. 

Positive  Life— Services  such  as  one-on-one  counseling  and  "street  intercepts"  for  HIV- 
positive  men  to  address  HIV  risk  behavior,  substance  abuse  and  mental  health  issues. 
Circle  of  Care  Collaboration — Working  together,  Shanti,  the  Black  Coalition  on  AIDS  and 
Instituto  Familiar  de  la  Raza  provide  case  management,  peer  advocacy  and  treatment 
advocacy  to  HIV-positive  people  who  are  uninsured,  underinsured  or  have  severe  needs. 
Emotional  and  Practical  Support — Services  such  as  intake  and  needs  assessments,  referrals 
to  off-site  services,  and  one-to-one  practical  and  emotional  support  for  low-income  and 
homeless  people  with  AIDS. 

St.  Mary's  Integrated  HIV  Services — Provides  peer  advocacy  for  HIV-positive  people  and 
management  supervision  under  a  subcontract  with  St.  Mary's  Medical  Center. 
HIV  Health  Services  Planning  Council  Support — Supports  the  administrative  activities  of 
the  council,  a  body  that  is  mandated  by  the  federal  Ryan  White/CARE  Act. 

Six  of  the  reports  gave  overall  program  ratings  of  commendable/exceeds  standards,  while  one  did 
not  include  ratings.  We  interviewed  DPH  staff  including  program  officers  Diane  Carr,  Joseph 
Cecere,  Hilda  Jones  and  John  Pabustan,  who  had  positive  comments  about  Shanti's  programs. 

Tenderloin  Housing  Clinic  (THC) 

THC  leases  and  operates  permanent,  affordable  supportive  housing  primarily  for  homeless  single 
adults  and  provides  a  law  office  that  gives  free  assistance  to  tenants.  As  of  June  2005,  THC  held 
$1 1 .4  million  in  contracts  with  HSA,  of  which  $8.5  million  was  for  fiscal  year  2004-05. 

THC  appears  to  have  adequate  internal  financial  controls  to  ensure  that  it  is  properly  administering 
its  contracts  with  the  HSA  and,  according  to  HSA,  is  performing  satisfactorily  under  these 
contracts.  We  interviewed  THC  staff  and  examined  documents  prepared  by  THC  concerning  areas 
including  its  building  leases,  federal  taxes,  and  payroll  procedures.  We  also  reviewed  documents 
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prepared  by  THC's  independent  auditor  to  support  its  conclusions  concerning  THC's  internal 
controls. 

We  reviewed  the  audited  financial  statements  of  THC  for  the  period  of  July  1,  2003  to  June  30, 
2004,  and  found  that  THC's  independent  auditor  rendered  an  unqualified  opinion  and  a  report  on 
internal  controls  over  compliance  with  the  requirements  of  laws,  regulations,  contracts,  and  grants 
applicable  to  each  of  its  major  federal  programs  that  showed  no  material  compliance  weaknesses. 
The  independent  auditor  did  note,  however,  a  material  financial  statement  weakness:  THC's 
accounting  system  captured  financial  information  on  a  cash  basis  rather  than  an  accrual  basis 
during  the  audit  period.  The  effect  of  this  was  that,  although  THC's  financial  system  supported 
what  was  billed  to  HSA,  it  did  not  ensure  that  the  financial  statements  reflected  the  full  costs 
incurred  to  run  the  organization  during  the  accounting  period  reported  on.  Because  we  made  our 
review  to  determine  if  THC  billed  the  City  for  actual  costs  that  were  appropriately  assigned  to  each 
cost  center  that  the  City  reimburses,  the  possibility  that  there  were  additional  expenses  at  the  end 
of  one  year  that  were  not  be  billed  until  the  following  year  did  not  appear  to  us  to  be  a  significant 
problem.  This  is  because  THC  is  an  otherwise  financially  healthy  organization.  According  to 
THC's  chief  financial  officer,  THC  has  put  great  effort  into  correcting  the  weakness  noted  by  the 
independent  auditor  and  can  now  generate  accurate  financial  statements  while  satisfying  City 
invoicing  practices,  compliance  issues  and  Generally  Accepted  Accounting  Principles. 

As  part  of  our  evaluation  of  internal  controls,  we  reviewed  the  audit  workpapers  of  THC's 
independent  auditor.  We  determined  that  the  auditor  had  engaged  in  adequate  planning  and  had 
gained  an  understanding  and  performed  sufficient  tests  of  transactions  to  conclude  that  internal 
controls  were  in  place. 

We  reviewed  three  recent  monitoring  reports  prepared  by  HSA,  each  concerning  a  different  major 
THC  program: 

•  Housing  First  (Master  Lease) — HSA's  Housing  Access  Team,  emergency  shelters  and 
other  qualified  agencies  refer  tenants  to  THC,  which  leases  and  operates  12  buildings  as 
supportive  housing  for  formerly  homeless  people. 

•  Modified  Payment — Housing  placement  services  for  single  adults  in  residential  hotels. 
Participants  have  their  income  support  checks  sent  directly  to  THC,  which  automatically 
deducts  the  tenant  contribution  and  pays  the  rent. 

•  CalWORKS  Housing  Subsidy— Placed  working,  CalWORKS-enrolled  families  into 
private  apartments,  then  offered  them  a  diminishing  three-year  rent  subsidy,  move-in  costs 
and  ongoing  support  services.  According  to  HSA,  the  program  served  89  clients  but  was 
discontinued  after  June  30,  2005. 

We  found  that  all  three  of  the  reports  gave  overall  ratings  of  comrnendable/exceeds  standards. 
Further,  we  interviewed  HSA  staff  including  program  monitors  Dion  Roberts  and  Scott  Walton, 
who  had  positive  comments  about  THC's  programs.  Although  not  yet  complete  at  the  time  of  our 
fieldwork,  HSA  monitoring  for  fiscal  year  2004-05  indicates  that  THC  programs  again  met  or 
exceeded  standards. 

Westside  Community  Mental  Health  Center  (Westside) 


Westside  provides  family-centered  mental  health  services,  drug  abuse  prevention  and  treatment, 
HIV-related  services  and  social  services  with  particular  emphasis  on  African  Americans  and  other 
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underserved  populations  of  all  ages  in  the  Western  Addition  neighborhood.  As  of  June  2005, 
Westside  held  $15.2  million  in  contracts  with  DPH,  of  which  $10.1  million  was  for  fiscal  year 
2004-05. 

We  reviewed  the  audited  financial  statements  of  Westside  for  the  period  of  July  1,  2003  to  June  30, 
2004,  and  found  that  Westside' s  independent  auditor  rendered  an  unqualified  opinion  and  a  report 
on  compliance  and  internal  control  over  financial  reporting  that  noted  no  material  weaknesses. 
Based  on  Westside's  audited  financial  statements,  we  documented  key  indicators  of  its  fiscal 
health  and  found  that  Westside  appears  to  be  financially  healthy. 

We  also  reviewed  11  recent  annual  monitoring  reports  prepared  by  DPH,  each  concerning  a 
different  Westside  program  in  San  Francisco: 

•  ACT  (Assertive  Community  Treatment) — Intensive  mental  health  outpatient  services  based 
on  the  assertive  case  management  model.  Service  users  are  to  participate  in  program 
development. 

•  AIDS  Home  Care — Provides  home/hospice/attendant  care  services  to  homebound,  multiply 
diagnosed  people  from  underserved  ethnic  minority  groups  who  have  HIV/ AIDS. 

•  Crisis  Clinic — Provides  crisis,  urgent  care,  and  case  management  services  to  reduce  the  use 
of  inpatient  psychiatric  hospital  services. 

•  HIV  Counseling,  Testing,  Referral  and  Partner  Notification — Clients  of  Westside's  alcohol 
and  drug  treatment  programs  are  referred  for  risk  assessment  and  offered  HIV  testing. 
Anonymous,  drop-in  HIV  testing  is  offered  to  the  general  public.  Services  include  risk 
reduction  counseling,  education  and  prevention,  and  case  management. 

•  Inner  City  Outpatient — Intensive  outpatient  services  at  a  South-of-Market  location  to  adults 
who  arc  impaired  physically,  socially  or  emotionally  or  who  have  legal  difficulties  because 
of  their  alcohol  or  drug  use. 

•  Institute  For  the  Arts  of  Living — A  partial-day  treatment  program,  with  bilingual 
(Spanish/English)  services,  and  mental  health  treatment  services  to  severely  emotionally 
disabled  children  and  adolescents  aged  1 1  to  18. 

•  Integrated  Child,  Youth  and  Family  Services — Child  and  adolescent  outpatient  services, 
substance  abuse  prevention,  mental  health  therapy  to  severely  emotionally  disabled  public 
school  students,  multicultural  mental  health  consultation,  and  training  and  direct  services  to 
improve  the  quality  of  child  care  services  provided  to  at-risk  children  and  their  families. 

•  Methadone  Maintenance  and  Detoxification — Maintenance  program  treats  opiate-addicted 
adults  with  daily  doses  of  methadone,  counseling  and  case  management.  Detoxification  is  a 
short-term  (21 -day)  service. 

•  New  Directions  Outpatient — Offers  educational  groups,  individual  sessions,  case 
management  and  medication  support  to  adults  with  serious  alcohol  and/or  drug  problems  at 
the  same  location  as  the  Inner  City  Outpatient  program. 

•  Westside  CalWORKS  Counseling  Services — Any  combination  of  mental  health  and 
substance  abuse  treatment  services  to  welfare  recipients  and  their  family  members  who  are 
enrolled  in  San  Francisco's  CalWORKS  (welfare-to-work)  program. 

•  Women's  HIV  Prevention — HIV  educational  and  prevention  drop-in  groups,  venue-based 
individual  outreach,  and  individual  risk-reduction  counseling  with  a  focus  on  African- 
American  females,  a  high-risk  sub-population. 

One  report  gave  an  average  rating  of  commendable/exceeds  standards,  eight  reports  had  an 
average  rating  of  acceptable/meets  standards,  one  program  (HIV  Counseling,  Testing,  Referral  and 
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Partner  Notification)  received  an  average  rating  of  improvement  needed/below  standards,  and  one 
(Women's  HIV  Prevention)  had  an  overall  rating  of  unacceptable.  We  interviewed  DPH  staff 
including  program  officers  Edwin  Batongbacal,  Erik  Dubon,  Mario  Hernandez  and  Hilda  Jones, 
and  discussed  Westside's  programs. 

The  program  that  received  the  improvement  needed  rating  for  fiscal  year  2003-04  primarily 
received  that  rating  because  the  program  provided  only  70%  of  the  contracted  units  of  service  and 
served  only  62%  of  the  contracted  number  of  unduplicated  clients.  According  to  Westside,  this 
program  had  staff  turnover  and  has  since  stabilized  and  improved.  The  Women's  HIV  Prevention 
program  had  an  overall  rating  of  unacceptable  for  the  first  six  months  of  the  contract  (July- 
December  2004)  because  it  had  not  delivered  the  contracted  number  of  units  of  service.  Per 
Westside,  this  occurred  due  to  considerable  staff  turnover  and  poor  client  participation.  Westside  is 
not  providing  Women's  HIV  prevention  services  at  this  time.  Westside's  proposal  to  operate  this 
program  for  fiscal  year  2005-06  was  unsuccessful. 

The  City  Can  Improve  Performance  Measures  in  Its  Health  and  Human  Services  Contracts 

We  reviewed  numerous  contracts  between  the  City  and  the  six  nonprofit  service  providers,  ail  of 
which  have  a  scope  of  work  section  (Appendix  A)  that  usually  includes  performance  objectives  for 
the  work.  In  particular,  we  analyzed  the  performance  sections  of  the  two  large  mental  health 
contracts  RAMS  held  with  DPH  in  fiscal  year  2004-05.  Eleven  programs  and  a  total  of  92 
performance  objectives  are  included  in  these  two  contracts.  We  focused  on  the  performance 
objectives,  which  DPH  created,  and  found  a  mixed  bag:  some  were  fairly  stated  objectives  and 
others  well-formulated  performance  measures,  but  too  many  lacked  quantitative  targets  that  would 
make  them  legitimate  performance  measures,  or  had  other  weaknesses.  Our  findings  include  the 
following. 

•  Many  objectives  have  no  corresponding  performance  measures.  As  stated  above,  of  the 
92  performance  objectives  stated  in  RAMS's  contracts  for  adult  services  (5  programs)  and 
children's  services  (6  programs),  almost  half  (45),  in  our  opinion,  were  not  performance 
measTrfe^That  i"s,"  "m^y'were- not'state^'lis^'a  measurable  achievement  with  a  quantitative 
target  for  the  contractor  to  meet.  Having  objectives  in  a  contract  is  fine,  but  each  objective 
should  have  at  least  one  corresponding  performance  measure.  Performance  objectives 
without  measurable  targets  are  difficult  or  impossible  for  the  City  to  monitor  and  for  the 
contractor  to  demonstrate  that  it  achieved.  Although  the  contracts  note  that  85% 
compliance  meets  standard  for  all  objectives  unless  otherwise  stated,  a  percentage  is 
impossible  to  apply  to  some  of  the  objectives  included. 

•  Process-oriented  compliance  requirements  are  presented  as  performance  objectives. 

We  found  numerous  examples  where  the  City  included  procedural  or  administrative 
requirements  that  clutter  the  sections  of  the  contracts  that  should  be  dedicated  to  the 
programs'  key  objectives  and  the  corresponding  measures  by  which  the  providers' 
performance  will  be  assessed.  For  example,  for  RAMS's  Children  Outpatient  and 
Prevention  Services,  the  contract  included  as  performance  objectives  that  RAMS  must  have 
a  valid  legal  entity  number  and  must  establish  its  published  rates.  Although  these  may  be 
important  requirements,  they  are  not  performance  objectives  or  performance  measures  and 
do  not  belong  in  a  section  of  the  contract  that  should  be  devoted  to  the  key  program 
objectives,  such  as  units  of  service  and  client  outcomes,  and  how  the  provider's  success  or 
failure  to  attain  them  will  be  measured. 
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•  Some  measures  are  weak  and  few  address  outcomes.  Of  the  performance  measures  that 
are  in  City  contracts,  many  are  poorly  formulated.  Of  the  47  items  categorized  as 
performance  objectives  in  the  two  RAMS  contracts  that  we  considered  to  be  performance 
measures,  we  found  27  (57%)  to  have  problems  with  formulation,  including  vague,  or 
illogical  wording,  lack  of  relevance  to  the  program,  and  other  problems.  For  example,  for 
RAMS's  Employment  Services  Program  (Hire-Ability  Vocational  Services),  a  performance 
objective  in  the  contract  is  that  at  least  50%  of  clients  will  be  in  supportive  employments, 
volunteer  jobs,  and  schools  or  participate  in  self-help  groups.  As  worded,  it  is  unclear  how 
long  the  client  is  expected  to  stay  in  the  program  and  how  long  the  client  is  expected  to  stay 
in  employment,  a  volunteer  job,  school,  or  a  self-help  group.  In  addition,  this  measure  is 
bundled,  giving  several  options  for  the  client's  status  without  specifying  targets  for  any  one 
of  them.  Also,  it  is  unclear  whether  all  of  the  options  are  equally  valuable  measures  of 
success  for  a  vocational  services  program.  Overall,  measures  of  service  quality,  client 
satisfaction  and  client  outcomes  are  unusual,  thus  giving  the  City  a  limited  basis  on  which 
to  assess  program  effectiveness.  In  fact,  of  the  47  statements  in  the  two  RAMS  contracts 
that  we  consider  to  be  performance  measures,  only  15  (32%)  address  program  outcomes. 

•  DPH  contract  monitor  acknowledges  that  objectives  in  contracts  are  insufficient.  A 

DPH  contract  monitor  we  interviewed  noted  that  the  performance  objectives  in  the 
contracts  are  not  enough  to  measure  the  effectiveness  of  RAMS's  programs.  He  also  stated 
that,  in  general,  objectives  are  intended  to  promote  department  policy,  not  to  verify 
performance,  so  DPH  does  not  check  or  verify  the  data,  which  is  mostly  self-reported  by 
the  providers.  This  calls  into  question  the  value  of  including  performance  measures  in  City 
contracts. 

Conclusion 

The  City  spends  more  than  $500  million  annually,  and  has  contracts  with  over  600  nonprofit 
agencies  that  provide  a  wide  array  of  human  services.  Given  the  critical  importance  of  this  area, 
and  the  overall  San  Francisco  Charter  requirement  that  the  City  Services  Auditor  audit  and  report 
on  the  level  and  effectiveness  of  public  services  (Appendix  F),  we  anticipate  that  the  City  Services 
Auditor  will  carry  out  audits  of,  and/or  provide  some  form  of  technical  assistance  to,  nonprofit 
contractors  in  every  fiscal  year.  We  welcome  comments  and  questions  on  this  report  and  on  our 
overall  efforts  in  this  area. 

Mark  Tipton,  Deborah  Gordon,  Lorita  Chung,  Donna  Crume  and  John  Haskell  of  my  staff 
conducted  this  review.  I  am  available  to  answer  any  questions  you  may  have  about  our  findings  or 
recommendations  at  415-554-7522  or  Peg.Stevenson@sfgov.org. 

cc:         David  Curto,  Human  Services  Agency 
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